Information processing apparatus, imaging apparatus, method, and storage medium

ABSTRACT

There is provided with an information processing apparatus. An acquisition unit acquires a captured image. A human body detection unit detects a human body area from the image. A first exposure determination unit determines exposure based on luminance of the human body area detected by the human body detection unit. A face detection unit detects a face area from an image captured at the exposure determined by the first exposure determination unit. A second exposure determination unit determines exposure based on luminance of the face area detected by the face detection unit and to maintain the exposure until a predetermined condition is satisfied.

BACKGROUND OF THE INVENTION Field of the Disclosure

The present invention relates to an information processing apparatus, animaging apparatus, a method, and a storage medium.

Description of the Related Art

A technology that detects a face area of an object from a captured imageand adjusts image quality of the captured image based on informationabout the face area has been known (see Japanese Patent ApplicationLaid-Open No. 2017-092544).

SUMMARY OF THE DISCLOSURE

According to one embodiment of the present disclosure, an informationprocessing apparatus is provided and includes an acquisition unitconfigured to acquire a captured image; a human body detection unitconfigured to detect a human body area from the image; a first exposuredetermination unit configured to determine exposure based on luminanceof the human body area detected by the human body detection unit; a facedetection unit configured to detect a face area from an image capturedat the exposure determined by the first exposure determination unit; anda second exposure determination unit configured to determine exposurebased on luminance of the face area detected by the face detection unitand to maintain the exposure until a predetermined condition issatisfied.

According to another embodiment of the present disclosure, a method isprovided and includes acquiring captured image; detecting a human bodyarea from the image; determining exposure based on luminance of thedetected human body area; detecting a face area from an image capturedat the determined exposure; and determining exposure based on luminanceof the detected face area and maintaining the exposure until apredetermined condition is satisfied.

Further features of the present invention will become apparent from thefollowing description of embodiments with reference to the attacheddrawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram illustrating a configuration of an imagingcontrol system according to a first embodiment of the present invention.

FIG. 2 is a block diagram illustrating an internal configuration of amonitoring camera according to the first embodiment.

FIG. 3 is a block diagram illustrating an internal configuration of aclient apparatus according to the first embodiment.

FIG. 4 is a diagram illustrating functions and configurations performedby the client apparatus.

FIG. 5 is a flowchart illustrating exposure control processing accordingto the first embodiment.

FIG. 6 is a flowchart illustrating exposure control processing accordingto a second embodiment.

FIGS. 7A and 7B are diagrams respectively illustrating a face detectionarea and a human body detection area.

FIGS. 8A and 8B are a flowchart illustrating exposure control processingaccording to a third embodiment.

FIG. 9 is a flowchart illustrating exposure control processing accordingto a fourth embodiment.

DESCRIPTION OF THE EMBODIMENTS

Embodiments of the present invention are described in detail below withreference to the drawings. The following embodiments do not limit thepresent invention, and all of combinations of features described in theembodiments are not necessarily essential for solving means of thepresent invention. Configurations of the embodiments can beappropriately corrected or modified depending on a specification of anapparatus to which the present invention is applied and various kinds ofconditions (use condition, use environment, etc.). The technical scopeof the present invention is defined by claims and is not limited by thefollowing individual embodiments. Further, parts of the embodimentsdescribed below may be appropriately combined with one another.

One or more of functional blocks illustrated in the drawings describedbelow may be implemented by hardware such as an application specificintegrated circuit (ASIC) and a programmable logic array (PLA), or maybe implemented through execution of software by a programmable processorsuch as a central processing unit (CPU) and a micro processing unit(MPU).

Further, one or more of the functional blocks may be implemented by acombination of software and hardware. Thus, in the followingdescription, even in a case where different functional blocks aredescribed as main executors of operation, the functional blocks can beimplemented by the same hardware as the main executor.

First Embodiment (Basic Configuration)

FIG. 1 is a diagram illustrating a configuration of an imaging controlsystem 100 according to a first embodiment of the present invention.

The imaging control system 100 includes a monitoring camera 101, anetwork 102, a client apparatus 103, an input device 104, and a displaydevice 105. The monitoring camera 101 is an imaging apparatus thatacquires a moving image and can perform imaging of an object and imageprocessing. The monitoring camera 101 and the client apparatus 103 arecommunicably connected to each other via the network 102. The clientapparatus 103 is communicably connected with the input device 104 andthe display device 105. The client apparatus 103 is an apparatus thatprocesses various information (information processing apparatus). Theclient apparatus 103 also functions as an imaging control apparatus thatcontrols imaging by the monitoring camera 101.

The input device 104 includes a mouse and/or a keyboard, and is operatedby a user of the client apparatus 103.

The display device 105 is a device including a monitor that displays animage received from the client apparatus 103. The display device 105 canalso function as a user interface (UT) such as a touch panel. In thiscase, the display device 105 also functions as an input device forinputting an instruction, information, data, etc. to the clientapparatus 103.

In FIG. 1, the client apparatus 103, the input device 104, and thedisplay device 105 are illustrated as apparatuses independent of oneanother; however, the present embodiment is not limited to such aconfiguration. For example, the client apparatus 103 and the displaydevice 105 may be integrated, or the input device 104 and the displaydevice 105 may be integrated. Alternatively, the client apparatus 103,the input device 104, and the display device 105 may be integrated. In acase where the client apparatus 103 and the display device 105 areintegrated, an integrated apparatus takes a form of, for example, apersonal computer, a tablet terminal, or a smartphone.

(Configuration of Monitoring Camera)

FIG. 2 is a block diagram illustrating an internal configuration of themonitoring camera 101. The monitoring camera 101 includes an imagingoptical system 201, an imaging device 202, a camera CPU 203, a read onlymemory (ROM) 204, a random access memory (RAM) 205, an imaging systemcontrol unit 206, a communication control unit 207, an analog/digital(A/D) conversion unit 208, an image processing unit 209, an encoder unit210, and a network interface (I/F) 211. The units (203 to 211) of themonitoring camera 101 are connected to one another via a system bus 212.

The imaging optical system 201 is an optical component group thatincludes a zoom lens, a focus lens, a shake correction lens, adiaphragm, and a shutter, and collects optical information about theobject. The imaging optical system 201 is connected to the imagingdevice 202.

The imaging device 202 is a charge-accumulation solid-state imagingdevice such as a complementary metal-oxide semiconductor (CMOS) imagingdevice and a charge-coupled device (CCD) imaging device that converts alight flux collected by the imaging optical system 201 into a currentvalue (signal value). The imaging device 202 acquires color informationin combination with a color filter or the like. The imaging device 202is connected to the A/D conversion unit 208.

The camera CPU 203 is a control unit that integrally controls operationof the monitoring camera 101. The camera CPU 203 reads a command storedin the ROM 204 or the RAM 205 and performs processing based on a resultof the reading.

The imaging system control unit 206 controls the components of themonitoring camera 101 based on an instruction from the camera CPU 203.For example, the imaging system control unit 206 performs control suchas focus control, shutter control, and diaphragm control in the imagingoptical system 201.

The communication control unit 207 performs control to transmit, to thecamera CPU 203, a control command (control signal) from the clientapparatus 103 to each of the components of the monitoring camera 101through communication with the client apparatus 103.

The A/D conversion unit 208 converts a light quantity of the objectdetected by the imaging device 202 into a digital signal (image data).The A/D conversion unit 208 transmits the digital signal to the imageprocessing unit 209.

The image processing unit 209 performs image processing on the imagedata including the digital signal received from the imaging device 202.The image processing unit 209 is connected to the encoder unit 210.

The encoder unit 210 performs processing to convert the image dataprocessed by the image processing unit 209 into data in a file formatsuch as Motion JPEG, H.264, or H.265. The encoder unit 210 is connectedto the network I/F 211.

The network I/F 211 is an interface used for communication with anexternal apparatus such as the client apparatus 103 via the network 102,and is controlled by the communication control unit 207.

The network 102 is an internet protocol (IP) network connecting themonitoring camera 101 with the client apparatus 103. The network 102includes a plurality of routers, switches, and cables compatible with acommunication standard such as Ethernet®. In the present embodiment, thecommunication standard, the scale, and the configuration of the network102 are not limited as long as the network 102 allows for thecommunication between the monitoring camera 101 and the client apparatus103. Examples of the network 102 include the Internet, a wired localarea network (LAN), a wireless LAN, and a wide area network (WAN).

(Configuration of Client Apparatus)

FIG. 3 is a block diagram illustrating an internal configuration of theclient apparatus 103.

The client apparatus 103 includes a client CPU 301, a main storagedevice 302, an auxiliary storage device 303, an input I/F 304, an outputI/F 305, and a network I/F 306. The elements of the client apparatus 103are communicably connected with one another via a system bus 307.

The client CPU 301 is a central processing unit integrally controllingoperation of the client apparatus 103. The client CPU 301 may integrallycontrol the monitoring camera 101 via the network 102.

The main storage device 302 is a storage device such as a RAM thatfunctions as a temporary storage area for data for the client CPU 301.For example, the main storage device 302 stores in advance a pattern(pattern corresponding to a face feature portion and a human bodyfeature portion) for pattern matching used when the client apparatus 103performs face detection and human body detection.

The auxiliary storage device 303 is a storage device such as a hard diskdrive (HDD), a ROM, or a solid state device (SSD) that stores variouskinds of programs and various kinds of setting data.

The input I/F 304 is an interface used when the client apparatus 103receives input (signal) from the input device 104 or other devices.

The output I/F 305 is an interface used when the client apparatus 103outputs information (signal) to the display device 105 or other devices.

The network I/F 306 is an interface used for communication with theexternal apparatus such as the monitoring camera 101 via the network102.

When the client CPU 301 performs processing based on the programs storedin the auxiliary storage device 303, functions and processing of theclient apparatus 103 illustrated in FIG. 4 are implemented. Details ofthe functions and the processing of the client apparatus 103 aredescribed below.

(Functions of Client Apparatus)

FIG. 4 is a diagram illustrating the functions performed by the clientapparatus 103. In other words, units (functional blocks) illustrated inFIG. 4 are functions that can be performed by the client CPU 301.

As illustrated in FIG. 4, the client CPU 301 of the client apparatus 103includes an input information acquisition unit 401, a communicationcontrol unit 402, an input image acquisition unit 403, a camerainformation acquisition unit 404, and a detection method setting unit405. The client CPU 301 further includes an object detection unit 406,an exposure determination unit 407, and a display control unit 408. Theclient apparatus 103 may perform the functions of the units 401 to 408illustrated in FIG. 4 by hardware (or software) different from theclient CPU 301.

The input information acquisition unit 401 receives input from the uservia the input device 104.

The communication control unit 402 performs control to receive an imagetransmitted from the monitoring camera 101 (image captured by monitoringcamera 101) via the network 102. The communication control unit 402further performs control to transmit a control command from the clientapparatus 103 to the monitoring camera 101 via the network 102.

The input image acquisition unit 403 acquires the image transmitted fromthe monitoring camera 101 via the communication control unit 402 as atarget image of object detection processing (an image to which objectdetection processing is applied). Details of the object detectionprocessing are described below.

The camera information acquisition unit 404 acquires camera information(imaging information) about the monitoring camera 101 capturing an imageof the object via the communication control unit 402. The camerainformation (imaging information) is various information related toimaging of the object and to acquire an image. Examples of the camerainformation include an exposure parameter such as an aperture value.

The detection method setting unit 405 sets a predetermined (appropriate)detection method for the image acquired by the input image acquisitionunit 403 among various detection methods including detection of a facearea (face detection) and detection of a human body area (human bodydetection). In a case where the face detection is performed, the objectdetection unit 406 described below preferentially detects a face area inthe image. In a case where the human body detection is performed, theobject detection unit 406 preferentially detects a human body area inthe image.

In the present embodiment, the detection method setting unit 405 sets(selects) the detection method that is the face detection or the humanbody detection. However, the present embodiment is not limited to suchsetting. For example, a detection method of detecting a partial featurearea of a person, such as an upper body, a head, and eyes, a nose, and amouth of the face, may be set (selected).

Further, in the present embodiment, the object to be detected is aperson; however, a specific area of a predetermined object other than aperson may be detected. The predetermined object preset by the clientapparatus 103, such as a face of an animal or an automobile, may bedetected.

The exposure determination unit 407 determines an exposure value basedon image information about the object area obtained from the objectdetection unit 406. The communication control unit 402 transmits(outputs) the above-described exposure value to the monitoring camera101. Then, the exposure control (exposure adjustment) is performed viathe communication control unit 207. A detailed processing flow relatingto the camera information acquisition unit 404, the detection methodsetting unit 405, the object detection unit 406, and the exposuredetermination unit 407 is described below with reference to a flowchartof FIG. 5.

The display control unit 408 outputs the captured image on which anexposure correction determined by the exposure determination unit 407 isreflected to the display device 105 in response to an instruction fromthe CPU 301.

(Object Detection Processing and Exposure Determination Processing)

The exposure control processing according to the present embodiment isdescribed below with reference to the flowchart of FIG. 5.

In the imaging control system 100 in FIG. 1, the monitoring camera 101,the client apparatus 103, the input device 104, and the display device105 have been turned on, and connection (communication) between themonitoring camera 101 and the client apparatus 103 has been established.

In this state, imaging of the object, transmission of image data fromthe monitoring camera 101 to the client apparatus 103, and image displayon the display device 105 are repeated at a predetermined updating cycleby the monitoring camera 101.

The processing of the flowchart in FIG. 5 is started by the client CPU301 in response to input of the captured image of the object from themonitoring camera 101 to the client apparatus 103 via the network 102.

First, in step S501, the detection method setting unit 405 sets facedetection to the object detection unit 406, and the object detectionunit 406 performs face detection processing on the input image. The mainstorage device 302 of the client apparatus 103 stores in advancepatterns corresponding to face feature portions and human body featureportions. The object detection unit 406 detects a face area byperforming pattern matching based on the patterns.

Next, in step S502, the object detection unit 406 determines whether aface area is detected in the image by the face detection processingperformed in step S501. In a case where no face area is detected (NO instep S502), the processing proceeds to step S503. In a case where atleast one or more face areas are detected (YES in step S502), theprocessing proceeds to step S505.

In step S503, the detection method setting unit 405 sets human bodydetection to the object detection unit 406, and the object detectionunit 406 performs human body detection processing on the input image.

In the face detection, the face area cannot be accurately detected if adirection, a size, and brightness of the face are not suitable for theface detection, whereas in the human body detection, an area where aperson is present can be detected irrespective of the direction, thesize, and the brightness of the face.

In the human body detection according to the present embodiment, it isnot necessary to detect a whole body, and an upper body, a bust, or ahead area including a face may be detected. FIGS. 7A and 7B respectivelyillustrate a detection example of the face area and a detection exampleof the human body area.

In a case where a pattern matching method is adopted as the objectdetection method, a pattern (classifiers) created by using statisticallearning may be used as the pattern used in the pattern matching.Alternatively, the object may be detected by a method other than thepattern matching. For example, the object may be detected by usingluminance gradient in a local area. In other words, the object detectionmethod is not limited to a specific detection method, and variousmethods such as a detection method based on machine learning and adetection method based on distance information can be adopted.

In step S504, the object detection unit 406 determines whether a humanbody area is detected in the image by the human body detectionprocessing performed in step S503. In a case where no human body area isdetected (NO in step S504), the processing ends. In a case where thehuman body area is detected (YES in step S504), the processing proceedsto step S506.

In step S505, the exposure determination unit 407 sets the face areadetected in step S501 as an area of interest to be used in subsequentstep S507.

In step S506, the exposure determination unit 407 sets the human bodyarea detected in step S503 as the area of interest to be used insubsequent step S507.

In step S507, the exposure determination unit 407 calculates an averageluminance value of areas of interest set in step S505 or step S506. Morespecifically, the exposure determination unit 407 applies informationabout the number of areas of interest (number of detected faces or humanbodies) set in step S505 or step S506, positions of the areas ofinterest, and sizes of the areas of interest, to an equation (1)described below.

$\begin{matrix}{{\overset{\_}{I}}_{object} = {\frac{1}{f}{\sum\limits_{s = 1}^{f}\left\{ {\frac{1}{k_{s} \times l_{s}}{\sum\limits_{i = {{- k_{s}}/2}}^{k_{s}/2}{\sum\limits_{j = {{- l_{s}}/2}}^{l_{s}/2}{I\left( {{v_{s} + i},{h_{s} + j}} \right)}}}} \right\}}}} & (1)\end{matrix}$

In the equation (1), I(x, y) represents a luminance value at atwo-dimensional coordinate position (x, y) in a horizontal direction(x-axis direction) and a vertical direction (y-axis direction) in theimage. In addition, f represents the number of areas of interest, (v, h)represents a central coordinate of each of the areas of interest, krepresents a size of each of the areas of interest in the horizontaldirection, and l represents a detected size of each of the areas ofinterest in the vertical direction.

In step S508, the exposure determination unit 407 determines an exposurevalue EV_(correction) as an exposure target value based on the averageluminance value calculated in step S507. First, the exposuredetermination unit 407 calculates a difference value between an averageluminance value I_(object) of the areas of interest calculated in stepS507 and a target luminance value I_(object target) of each area ofinterest as represented by an equation (2). For example, the targetluminance value I_(object target) of each area of interest may bearbitrary set by the user or set to a value that improves accuracy bycomparing with authentication accuracy in face authentication.

ΔDiff=I _(object target) −Ī _(object)  (2)

Next, the exposure determination unit 407 determines the exposure valueEV_(correction) as represented by an equation (3). EV_(current) is anexposure value in additive system of photographic exposure (APEX)conversion based on an object luminance value (brightness value), and isset based on a program diagram relating to exposure control stored inadvance in the client apparatus 103.

$\begin{matrix}{{EV}_{correction} = \left\{ \begin{matrix}{{EV}_{current} - \beta} & {{{if}\mspace{14mu}\Delta\;{Diff}} < {- {Th}}} \\{EV}_{current} & {{{if}\mspace{14mu} - {Th}} \leqq {\Delta\;{Diff}} \leqq {Th}} \\{{EV}_{current} + \beta} & {{{if}\mspace{14mu}{Th}} < {\Delta\;{Diff}}}\end{matrix} \right.} & (3)\end{matrix}$

In the equation (3), a parameter β is a coefficient that influences acorrection degree (speed) when exposure is corrected to an under side oran over side of the current exposure value EV_(current). If the value ofthe parameter β is set large, processing speed (or time) until theexposure value reaches the exposure target value is increased; however,in a case where erroneous determination occurs in a detection result orin a case where detection of the object is unstable, brightness of theentire screen sharply changes. In contrast, if the value of theparameter β is set small, the processing speed (or time) until theexposure value reaches the exposure target value is decreased; however,the processing is robust to erroneous detection and the imagingcondition. The parameter β is set as a correction value of the exposureto the current exposure value EV_(current) in a case where a differencevalue ΔDiff is greater than or equal to a set threshold Th.

In step S509, the camera information acquisition unit 404 acquiresinformation on the current exposure value from the monitoring camera101.

In step S510, the exposure determination unit 407 communicates with themonitoring camera 101 via the communication control unit 402, and setsthe exposure value calculated in step S508 to the monitoring camera 101.

In step S511, the object detection unit 406 performs the face detectionprocessing in a manner similar to step S501.

In step S512, the object detection unit 406 determines whether a facearea has been detected in the image by the face detection processingperformed in step S511. In a case where a face area has been detected(YES in step S512), the processing proceeds to step S513. In a casewhere no face area has been detected (NO in step S512), the processingproceeds to step S514.

In step S513, the exposure determination unit 407 communicates with themonitoring camera 101 via the communication control unit 402, and setsthe monitoring camera 101 to maintain (fix) the exposure value set instep S510. Then, the processing ends. More specifically, the exposurevalue is maintained until a face or a human body is detected again andthe exposure target value is newly set.

In step S514, the exposure determination unit 407 communicates with themonitoring camera 101 via the communication control unit 402, andreturns the exposure target value to the exposure value (exposure setvalue) acquired in step S509. Then, the processing ends. In other words,the exposure value set in step S510 is discarded. Then, the exposurecontrol is performed while the exposure target value is successivelyupdated based on the luminance value of the entire image. The luminancevalue is not necessarily the luminance value of the entire image, andluminance values of one or more areas (predetermined areas) set inadvance in the image may be used. As described above, the imagingcontrol system 100 according to the present embodiment performs theexposure control based on the face area or the human body area. In thecase where the face is detected as a result, the exposure value ismaintained. Maintaining the exposure value in the above-described mannermakes it possible to maintain the exposure suitable for the facedetection. For example, even in a state of strong backlight at a storeentrance or a stadium gate using a glass door, it is possible tomaintain the exposure value suitable for the face detection.Accordingly, it is possible to quickly capture a face of an object(person) such as a customer or a visitor.

In the present embodiment, the exposure control is performed based onthe face area or the human body area. In the case where the face is notdetected as a result, the exposure value is returned to the originalexposure value. Thus, even in the case where the appropriate exposurecontrol is not performed due to erroneous detection of the face or thehuman body, the exposure value is not maintained.

Second Embodiment

A second embodiment of the present invention is described below withreference to FIG. 6 and FIGS. 7A and 7B. In the present embodiment, amethod of performing the exposure control more suitable for recognitionof the face is described. After the exposure control is performed basedon the human body area according to the first embodiment, the exposurecontrol is performed again based on the face area.

Configurations and processing similar to the configurations and theprocessing according to the first embodiment are denoted by the samereference numerals, and detailed descriptions thereof are omitted.

FIG. 6 is a flowchart according to the present embodiment. Hereinafter,exposure control processing according to the present embodiment isdescribed with reference to the flowchart illustrated in FIG. 6. Theprocessing start timing is similar to the processing start timing in thefirst embodiment. Thus, description thereof is omitted. Further, theprocessing in steps S501 to S514 is similar to the processing in stepsS501 to S514 in the first embodiment. Thus, description of theprocessing is omitted.

In step S601, the exposure determination unit 407 sets a human bodydetection flag representing detection of the human body to off. Theprocessing is performed in the case where it is determined in step S502that a face is detected. In and after subsequent step S505, the exposurecontrol is performed based on the face area as described in the firstembodiment.

In step S602, the exposure determination unit 407 sets the human bodydetection flag representing detection of the human body to on. Theprocessing is performed in a case where it is determined in step S504that a human body is detected. In and after subsequent step S506, theexposure control is performed based on the human body area as describedin the first embodiment.

In step S603, the exposure determination unit 407 determines whether thehuman body detection flag is off. In a case where the human bodydetection flag is off (YES in step S603), the processing proceeds tostep S513. Otherwise, namely, in a case where the human body detectionflag is on (NO in step S603), the processing proceeds to step S601. Inother words, since the human body detection flag is off in the casewhere the exposure control is performed based on the face area, theprocessing proceeds to step S513, and the exposure control (exposurevalue) is maintained. Then, the processing ends. In contrast, in thecase where the exposure control is performed based on the human bodyarea, the human body detection flag is on. Thus, the processing returnsto step S601, and the exposure control based on the face area isperformed. In this case, in step S505, the face area detected in stepS511 is set as the area of interest.

As illustrated in FIGS. 7A and 7B, the human body area may include anarea other than the face such as an area of clothing. Thus, in the casewhere the exposure value based on the human body area is maintained asin the first embodiment, although there are advantages that theprocessing is simple and responsiveness is excellent, the exposure maynot be necessarily suitable for the face area.

In the second embodiment, the exposure control is performed based on thehuman body area, and in the case where the face is detected as a result,the exposure control is performed again based on the face area.Performing the processing according to the present embodiment makes itpossible to set the exposure suitable for the face area. This allows formaintenance of the exposure more suitable for recognition of the face.

Third Embodiment

A third embodiment of the present invention is described below withreference to FIGS. 8A and 8B. As described above, in the firstembodiment and the second embodiment, the exposure value suitable forthe face detection can be maintained. However, for example, in a casewhere a face or a human body is not detected for a long time in a storehaving the small number of customers, and brightness inside and outsidethe store changes with time or due to weather or change of lighting, themaintained exposure value may not be suitable for the face detection. Inparticular, in a case where the brightness is largely changed so that abacklight state and a follow light state are switched, maintenance ofthe exposure value may cause overexposure or underexposure of the faceor the human body, which may largely deteriorate detection accuracy.

Thus, in the present embodiment, a method in which the maintenance ofthe exposure value is canceled based on the change in brightness of theimaging environment is described. Configurations and processing similarto the configurations and the processing according to the first andsecond embodiments are denoted by the same reference numerals, anddetailed descriptions thereof are omitted.

FIGS. 8A and 8B are a flowchart according to the present embodiment.Hereinafter, exposure control processing according to the presentembodiment is described with reference to the flowchart illustrated inFIGS. 8A and 8B. The processing start timing is similar to theprocessing start timing in the first embodiment. Thus, descriptionthereof is omitted. Further, the processing in steps S501 to S514 issimilar to the processing in steps S501 to S514 in the first embodiment.Thus, description of the processing is omitted.

In step S801, the exposure determination unit 407 sets an exposuremaintenance flag representing maintenance of the exposure control to on.The exposure maintenance flag is a static variable, and contents of theexposure maintenance flag are carried over to next exposure controlprocessing.

In step S802, the exposure determination unit 407 calculates an averageluminance value of the entire screen. The processing is performed in acase where the determination results in step S502 and in step S504 areboth no. In other words, the average luminance value of the entirescreen is calculated in a state where neither a face nor a human body isdetected and no object person is present.

In step S803, the exposure determination unit 407 determines whether theexposure maintenance flag is on. In a case where the exposuremaintenance flag is on (YES in step S803), the processing proceeds tostep S804. Otherwise (NO in step S803), the processing proceeds to stepS806.

In step S804, the exposure determination unit 407 stores the averageluminance value of the entire screen calculated in step S802 as areference luminance value (reference value). The reference luminancevalue is a static variable, and contents of the reference luminancevalue are carried over to the next exposure control processing.

In step S805, the exposure determination unit 407 sets the exposuremaintenance flag to off. Then, the processing ends.

On the other hand, in step S806, the exposure determination unit 407calculates a difference value between the average luminance value of theentire screen calculated in step S802 and the reference luminance valuestored in step S804. In the present embodiment, in a case where theprocessing in step S804 has never been performed before, the differencevalue is zero.

In step S807, the exposure determination unit 407 determines whether thedifference value calculated in step S806 is greater than or equal to apreset threshold. In a case where the difference value is greater thanor equal to the preset threshold (YES in step S807), the processingproceeds to step S808. Otherwise (NO in step S807), the processing ends.The threshold is preferably set, for example, based on a range of theaverage luminance value detectable by the object detection unit 406 thatis determined by capturing in advance various brightness patterns of theface and the human body by the monitoring camera 101 while changingexposure setting and illumination.

In step S808, the exposure determination unit 407 communicates with themonitoring camera 101 via the communication control unit 402, and resetsthe exposure value of the monitoring camera 101 to a prescribed valueheld by the monitoring camera 101. Then, the processing ends. To resetthe exposure value, for example, the exposure value EV_(correction) inthe equation (3) may be set to zero.

As described above, in the third embodiment, the maintenance of theexposure value is canceled based on the change in brightness of theimaging environment. Performing the processing according to the presentembodiment makes it possible to reset the exposure value even in thecase where the brightness of the imaging environment is largely changed.This makes it possible to prevent a decrease in the detection accuracyof the face or the human body.

In the present embodiment, the example based on the first embodiment hasbeen described to simplify the description; however, the presentembodiment can also be combined with the second embodiment. In thiscase, the processing in step S801 is performed after the processing instep S513 in the second embodiment, and the processing in steps S802 toS808 is performed in the case where the determination result in stepS504 is no.

Further, in the present embodiment, the method using the averageluminance value of the entire screen has been described; however, themethod is not limited thereto. For example, a weighted average luminancevalue obtained by weighting each of the areas where the face or thehuman body is detected, or an average luminance value of areas near theareas where the face or the human body is detected may be used.

Fourth Embodiment

In the above-described embodiments, the example is described where theexposure value is maintained in the case where the face is detectedafter the exposure control. However, depending on the conditions such asa field angle (imaging range) of the camera and moving speed of theobject, the object may go out of the field angle (imaging range) beforethe face detection processing is performed. In this case, the exposurevalue of the exposure control performed based on luminance of the areaof interest (face or human body) is not maintained.

In a fourth embodiment, in a case where face confirmation processing isnot valid (invalid), the exposure value is maintained irrespective ofpresence/absence of the face detection. Configurations and processingsimilar to the configurations and the processing according to theabove-described embodiments are denoted by the same reference numerals,and detailed descriptions thereof are omitted.

FIG. 9 is a flowchart according to the present embodiment. Hereinafter,exposure control processing according to the present embodiment isdescribed with reference to the flowchart illustrated in FIG. 9. Theprocessing start timing is similar to the processing start timing in thefirst embodiment. Thus, description thereof is omitted. Further, theprocessing in steps S501 to S514 is similar to the processing in stepsS501 to S514 in the first embodiment. Thus, description of theprocessing is omitted.

In step S901, the exposure determination unit 407 determines whether theface confirmation processing is valid. The face confirmation processingrefers to the face detection processing performed on the image after theexposure control performed in step S510. In a case where the faceconfirmation processing is valid (YES in step S901), the processingproceeds to step S511, and the face confirmation processing is performedin steps S511 and S512. In contrast, in a case where the faceconfirmation processing is not valid (NO in step S901), the processingproceeds to step S513, and the exposure value is maintained (exposure isfixed). In other words, the exposure value is maintained in a case wherea predetermined condition is satisfied. The predetermined condition isthe case where the face confirmation processing is invalid (NO in stepS901), or the case where the face confirmation processing is valid andthe face area is detected (YES in step S512).

The face confirmation processing is preferably set invalid when theobject goes out of the field angle (imaging area) before the faceconfirmation processing is completed because of a condition such as aninstallation state of the camera, the field angle (imaging range) of thecamera, and the moving speed of the object. Otherwise, the faceconfirmation processing is preferably set valid.

The face confirmation processing can be automatically set based on aratio of presence/absence of the face detection in step S512. Forexample, in a case where a ratio of presence of the face detection issmall, the environment is determined as an environment in which theobject easily goes out of the field angle, and the face confirmationprocessing is set valid. Otherwise, the face confirmation processing isset invalid.

The exposure fixed in step S513 is maintained until a face or a humanbody is newly detected. In other words, the fixed exposure is canceledwhen a face or a human body is newly detected. The fixing of theexposure may be canceled after a predetermined time has elapsed fromstart of the fixing (maintenance) of the exposure. After the fixing ofthe exposure is canceled, the exposure value is calculated based onluminance of a predetermined area of the captured image (e.g., entireimage or area near center of image), and the exposure control isperformed.

OTHER EMBODIMENTS

In the above-described embodiments, the example in which the face or thehuman body is detected has been described; however, a detected object isnot limited thereto. A specific object, for example, an automobile and anumber plate thereof may be detected.

In the above-described embodiments, the example in which the exposurecontrol of the monitoring camera 101 is performed by the clientapparatus 103 via the network 102 has been described; however, theexposure control may be performed by the monitoring camera 101.

A program (software) implementing a part or all of the control and thefunctions of the above-described embodiments may be supplied to animaging apparatus or an information processing apparatus via a networkor various kinds of recording media. Further, a computer (or CPU or MPU)of the imaging apparatus or the information processing apparatus mayread out and execute the program. In this case, the program and thestorage medium storing the program constitute each of the embodiments ofthe present invention.

A part or all of the control of the embodiments may be implemented by acircuit (e.g., application specific integrated circuit (ASIC)) thatimplements one or more functions.

While the embodiments of the present invention have been describedabove, the present invention is not limited to the embodiments, andvarious modifications and alternations can be made within the scope ofthe present invention.

Embodiment(s) of the present disclosure can also be realized by acomputer of a system or apparatus that reads out and executes computerexecutable instructions (e.g., one or more programs) recorded on astorage medium (which may also be referred to more fully as a‘non-transitory computer-readable storage medium’) to perform thefunctions of one or more of the above-described embodiment(s) and/orthat includes one or more circuits (e.g., application specificintegrated circuit (ASIC)) for performing the functions of one or moreof the above-described embodiment(s), and by a method performed by thecomputer of the system or apparatus by, for example, reading out andexecuting the computer executable instructions from the storage mediumto perform the functions of one or more of the above-describedembodiment(s) and/or controlling the one or more circuits to perform thefunctions of one or more of the above-described embodiment(s). Thecomputer may comprise one or more processors (e.g., central processingunit (CPU), micro processing unit (MPU)) and may include a network ofseparate computers or separate processors to read out and execute thecomputer executable instructions. The computer executable instructionsmay be provided to the computer, for example, from a network or thestorage medium. The storage medium may include, for example, one or moreof a hard disk, a random-access memory (RAM), a read only memory (ROM),a storage of distributed computing systems, an optical disk (such as acompact disc (CD), digital versatile disc (DVD), or Blu-ray Disc (BD)™),a flash memory device, a memory card, and the like.

While the present disclosure has been described with reference toembodiments, it is to be understood that the disclosure is not limitedto the disclosed embodiments, but is determined by the scope of thefollowing claims.

This application claims the benefit of Japanese Patent Applications No.2020-045744, filed Mar. 16, 2020, and No. 2020-181986, filed Oct. 30,2020, which are hereby incorporated by reference herein in theirentirety.

What is claimed is:
 1. An information processing apparatus, comprising: an acquisition unit configured to acquire a captured image; a human body detection unit configured to detect a human body area from the image; a first exposure determination unit configured to determine exposure based on luminance of the human body area detected by the human body detection unit; a face detection unit configured to detect a face area from an image captured at the exposure determined by the first exposure determination unit; and a second exposure determination unit configured to determine exposure based on luminance of the face area detected by the face detection unit and to maintain the exposure until a predetermined condition is satisfied.
 2. The information processing apparatus according to claim 1, wherein the predetermined condition is that, after the exposure is determined by the second exposure determination unit, the human body area is detected again by the human body detection unit or the face area is detected again by the face detection unit.
 3. The information processing apparatus according to claim 1, wherein the second exposure determination unit is configured to, when a predetermined time passes in a state where neither the face area nor the human body area is detected after maintenance of the exposure is started, cancel the maintenance of the exposure and newly determine exposure based on luminance of a predetermined area of the captured image.
 4. The information processing apparatus according to claim 3, further comprising: a storage unit configured to store average luminance of the predetermined area of the image as a reference value in a case where neither the face area nor the human body area is detected by the face detection unit or the human body detection unit after the second exposure determination unit starts maintenance of the exposure; and a calculation unit configured to calculate a difference value between current average luminance of the predetermined area and the reference value in a case where, after the storage unit stores the reference value, neither the face area nor the human body area is detected by the face detection unit or the human body detection unit again, wherein the second exposure determination unit is configured to, in a case where the difference value calculated by the calculation unit is greater than or equal to a threshold, cancel the maintenance of the exposure and newly determine exposure based on luminance of the predetermined area of the captured image.
 5. The information processing apparatus according to claim 3, wherein the predetermined area is an entire area of the captured image.
 6. The information processing apparatus according to claim 3, wherein the predetermined area is an area near a center of the captured image.
 7. The information processing apparatus according to claim 1, wherein the first exposure determination unit is configured to determine the exposure so that a difference between the luminance of the human body area detected by the human body detection unit and target luminance falls within a predetermined range.
 8. The information processing apparatus according to claim 1, wherein the second exposure determination unit is configured to determine the exposure so that a difference between the luminance of the face area detected by the face detection unit and target luminance falls within a predetermined range.
 9. The information processing apparatus according to claim 1, wherein the human body area is any of a whole body, an upper body, a bust, and a head area including a face.
 10. An imaging apparatus comprising the information processing apparatus according to claim
 1. 11. A method comprising: acquiring captured image; detecting a human body area from the image; determining exposure based on luminance of the detected human body area; detecting a face area from an image captured at the determined exposure; and determining exposure based on luminance of the detected face area and maintaining the exposure until a predetermined condition is satisfied.
 12. The method according to claim 11, wherein the predetermined condition is that, after the exposure is determined, the human body area is detected again or the face area is detected again.
 13. The method according to claim 11 wherein, when a predetermined time passes in a state where neither the face area nor the human body area is detected after maintenance of the exposure is started, the maintenance of the exposure is canceled, and exposure is newly determined based on luminance of a predetermined area of the captured image.
 14. A non-transitory computer-readable storage medium storing a program for causing a computer to execute a method according to claim
 11. 